Information linking method, information viewer, information register, and information search equipment

ABSTRACT

On an information registering equipment, content of interest rendered by media, such as a video image of interest distributed by TV broadcasting is captured and displayed. Position/area on the image are defined with a mouse or the like to obtain the target position/area. Information to identify the content is obtained from the TV tuner. Reference information is obtained through input means or by retrieval from the server for reference information. The obtained information specifics are transmitted to an information search equipment that in turn stores then into its database. An information viewing equipment obtains the target position/area and information to identify the content of interest and transmits the thus obtained data to the information search equipment. The information search equipment matches that data with each data record in the database and transmits reference information to the information viewing equipment on which the reference information is displayed.

BACKGROUND OF THE INVENTION

The present invention relates to a method for linking a video imagerendered by TV broadcasting and other means to information related tothe video image and an information viewer, information register,information search equipment and billing method, based on such method.

On a computer network called the Internet, previously stored informationis accessible over the network and can be retrieved and rendered fromthe World Wide Web (WWW) which would be regarded as a huge informationdatabase system wherein related information is linked together.

For building a Web site on the WWW, a method that will be describedbelow is generally used. Information such as text, sound and images islinked by means of a hypertext language called HTML (Hyper Text MarkupLanguage) and described as information items on a Web page (home page)of a Web site and information as tags specifying hyperlinks to other Webpages is also described in HTML on the Web home page.

Not only bushiness users, but also general personal users can gainaccess to Web sites and pages by using browsing software (hereinafterreferred to as a browser) run on a personal computer (PC). For improvinguser facilities, such a search service (hereinafter referred to as adirectory service) is provided on the WWW that enables ready access totarget information on a Web site/page, according to a keyword or thelike entered.

With digital broadcasting using broadcasting satellites, it is possiblethat a plurality of information contents are linked up and that a TVprogram and information distributed on the Internet are linked up byusing scripting languages as extensions of the HTML function forbroadcasting, such as BML (Broadcast Markup Language), B-XML (BroadcasteXtensible Markup Language), and B-HTML (Broadcast HTML). Informationdescribed in these scripting languages is transmitted to users by meansof framing such as standardized data broadcasting.

SUMMARY OF THE INVENTION

If users (audience of TV programs) should use a function that enablesthe user, by selecting a part or all of a video image displayed on a TVreceiver when watching a TV program, to obtain information related tothe video image, it would be convenient for them. For example, if theaudience should select (click) a costume that an actress wears who actsthe heroine of a drama program on the air with a pointing device such asa mouse, reference information related to the costume, such as itssupplier name and price, would be displayed on the TV monitor screen. Torealize this function, however, we would be confronted with thefollowing problems:

A first problem is how to link a video image to its referenceinformation. Conventional search engines for the above-mentioneddirectory service perform search, based on a keyword entered. To applysuch engines, it is necessary to convert the video image to a keyword;for example, in the above costume example, the name of the costume andits supplier name must be obtained from the video image. This isgenerally difficult in most cases. Thus, it is necessary to provide anew information linking method that enables the video image of interestto directly link to its reference information.

A second problem is how to provide an information registering equipmentfor making the above new information linking method practicable; thatis, equipment that enables the provider of information related to thevideo image of interest (who is hereinafter referred to as theregistrar) to link the video image to its reference information by easyoperation.

A third problem is-how to provide an information viewing equipment formaking the above new information linking method practicable; that is,equipment that enables the information viewer to view the referenceinformation by selecting a part or all of the video image of interest.

A fourth problem is how to provide an information search equipment formaking the above new information linking method practicable; that is,equipment that is able to search for target information registeredthereon, according to the search request from the above informationviewing equipment and transmit the result of search to the informationviewing equipment.

A fifth problem is how to provide a new billing method for making theservice of linking the video image of interest to its referenceinformation feasible as a business.

With the aim of solving the foregoing problems, the object of thepresent invention is to provide an information linking method,information viewing apparatus, information registering apparatus,information search apparatus, and billing method.

In order to solve the foregoing problems and in accordance with oneaspect of the present invention, the invention provides an informationlinking method for linking content of interest rendered by media toinformation related to the content of interest (this information ishereinafter referred to as reference information), using an informationregistering equipment, an information viewing equipment, an informationsearch equipment, and a computer network. This information linkingmethod comprises the steps wherein the information registering equipmenttransmits a set of first content-identifying information to identify thecontent of interest and the reference information across the computernetwork to the information search equipment; the information searchequipment registers the received set of the first content-identifyinginformation and the reference information into a database; theinformation viewing equipment transmits second content-identifyinginformation to identify the content of interest across the computernetwork to the information search equipment; and the information searchequipment matches the received second content-identifying informationwith the first content-identifying information registered in thedatabase and, if there is a match, transmits the reference informationfrom the set in which the reference information joined with the firstcontent-identifying information across the computer network to theinformation viewing equipment.

In the foregoing description of the information linking method, thecontent of interest rendered by media is information comprising animage. In this aspect, the information linking method comprises thesteps wherein the information registering equipment transmits firsttarget-defining information that designates a part or all of the imagein conjunction with the first content-identifying information across thecomputer network to the information search equipment; the informationsearch equipment registers the received first content-identifyinginformation, first target-defining information, and the referenceinformation in a set into the database; the information viewingequipment transmits second target-defining information that designates apart or all of the image in conjunction with the secondcontent-identifying information across the computer network to theinformation search equipment; and the information search equipmentmatches the received second content-identifying information with thefirst content-identifying information registered in the database and thereceived second target-defining information with the firsttarget-defining information registered in the database and, if there arematches for both couples of information, transmits the referenceinformation from the set in which the reference information joined withthe first content-identifying information and first target-defininginformation across the computer network to the information viewingequipment.

In the foregoing description of the information linking method, thecontent of interest rendered by media is video image informationdistributed by TV broadcasting and the content-identifying informationincludes a broadcasting channel number over which and time when thecontent was or will be broadcasted by the TV broadcasting, and moreover,a receiving area if the TV broadcasting is local TV broadcasting(restricted area broadcasting).

In the foregoing description of the information linking method, thereference information includes a URL (Uniform Resource Locator) thatdesignates a Web site/page on the Internet.

According to another aspect of the present invention, the inventionprovides an information viewing equipment which links content ofinterest rendered by media to its reference information. Thisinformation viewing equipment comprises means for capturing the contentof interest; means for obtaining content-identifying information toidentify the content of interest from the captured content of interest;means for transmitting the content-identifying information over acomputer network; means for receiving the reference information over thecomputer network; and means for outputting the received referenceinformation.

In the foregoing description of the information viewing equipment, thecontent of interest rendered by media is video image informationdistributed by TV broadcasting and the means for obtainingcontent-identifying information is to obtain at least a broadcastingchannel number over which and time when the content was or will bebroadcasted, and moreover, a receiving area if the TV broadcasting islocal TV broadcasting (restricted area broadcasting).

The information viewing equipment further comprises means for obtainingtarget-defining information that designates a part or all of a videoimage of interest distributed by the TV broadcasting and means fortransmitting the target-defining information joined with thecontent-identifying information over the computer network.

The information viewing equipment further comprises a time-shiftingapparatus for recording and reproducing the content of interest.

According to yet another aspect of the present invention, the inventionprovides an information registering equipment which links content ofinterest rendered by media to its reference information. Thisinformation registering equipment comprises means for capturing thecontent of interest; means for obtaining content-identifying informationto identify the content of interest; means for obtaining referenceinformation to join with the content-identifying information; and meansfor transmitting a set of the content-identifying information and thereference information over a computer network.

In the foregoing description of the information registering equipment,the content of interest rendered by media is video image informationdistributed by TV broadcasting and the means for obtainingcontent-identifying information is to obtain at least a broadcastingchannel number over which and time when the content was or will bebroadcasted, and moreover, a receiving area if the TV broadcasting islocal TV broadcasting (restricted area broadcasting).

The information registering equipment further comprises means forobtaining target-defining information that designates a part or all of avideo image of interest distributed by the TV broadcasting and means fortransmitting the target-defining information joined with thecontent-identifying information and the reference information over thecomputer network.

The information registering equipment further comprises a time-shiftingapparatus for recording and reproducing the content of interest.

According to a further aspect of the present invention, the inventionprovides an information search equipment which searches for referenceinformation linked with content of interest rendered by media. Thisinformation search equipment comprises means for inputting a set offirst content-identifying information and reference information receivedover a computer network; recording means for storing the set of thefirst content-identifying information and the reference information intoa database; means for inputting second content-identifying informationreceived over the computer network; an information matching apparatusfor comparing the first and second content-identifying informationspecifics for matching; and means for transmitting the referenceinformation from the set in which the reference information joined withthe first content-identifying information across the computer network tothe sender of the second content-identifying information (the sender ishereinafter referred to as a viewer) if there is a match as the resultof the matching.

In the foregoing description of the information search equipment, thecontent of interest rendered by media is video image informationdistributed by TV broadcasting. In this aspect, the information searchequipment includes means for inputting first target-defining informationreceived over the computer network, the first target-defininginformation designating a part or all of a video image of interestdistributed by the TV broadcasting, wherein the recording means storesthe first target-defining information and the first content-identifyinginformation in a set into the database. The information search equipmentfurther includes means for inputting second target-defining informationreceived over the computer network, the second target-defininginformation designating a part or all of a video image of interestdistributed by the TV broadcasting. The information matching apparatuscompares the first and second target-defining information specifics aswell as the first and second content-identifying information specificsfor matching.

The information search equipment further comprises means for controllingits operation such that, if there is a mismatch as the result ofcomparing the first and second content-identifying informationspecifics, the information search equipment sends the viewer notice thatreference information matching the target of search is not yetregistered as the result of search for matching.

The information search equipment further comprises means for storinginformation about the sender of the second content-identifyinginformation (in other words, the viewer) into the database and means fortransmitting the information about the viewer across the computernetwork to the sender of the first content-identifying information ifthere is a match as the result of comparing the first and secondcontent-identifying information specifics.

These and other objects, features and advantages of the presentinvention will become more apparent in view of the following detaileddescription of the preferred embodiments in conjunction withaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic structural drawing of a preferred embodiment ofthe present invention.

FIG. 2 shows a structural drawing of an information registeringequipment (102) and an information viewing equipment (103) embodiedaccording to the preferred embodiment of the present invention.

FIG. 3 shows a structural drawing in another form of the informationregistering equipment (102) and the information viewing equipment (103)embodied according to the preferred embodiment of the present invention.

FIG. 4 illustrates a data record example used for information searchaccording to the present invention.

FIG. 5 illustrates an example of matching operation by an informationmatching apparatus (110) used in the present invention.

FIG. 6 illustrates an example of information linking operation accordingto the present invention.

FIG. 7 illustrates another example of information linking operationaccording to the present invention.

FIG. 8 illustrates an information example that is displayed on theinformation viewing equipment (103) of the present invention.

FIG. 9 illustrates another information example that is displayed on theinformation viewing equipment (103) of the present invention.

FIG. 10 illustrates an example of post-processing of information linkingoperation according to the present invention.

FIG. 11 shows a system structural diagram of illustrative embodiment ofimplementing the post-processing of information linking operationaccording to the present invention.

FIG. 12 illustrates another example of a data record used forinformation search according to the present invention.

FIG. 13 illustrates yet another data record example used for informationsearch according to the present invention.

FIG. 14 illustrates a further data record example used for informationsearch according to the present invention.

FIG. 15 shows a structural diagram illustrating another illustrativeembodiment of the information search equipment (109) of the presentinvention.

FIG. 16 illustrates an example of defining the value of information usedfor the billing method of the present invention.

FIG. 17 illustrates yet another information example that is displayed onthe information viewing equipment (103) of the present invention.

FIG. 18 shows a schematic structural drawing of another preferredembodiment of the present invention.

FIG. 19 illustrates a further information example that is displayed onthe information viewing equipment (103) of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

With reference to the accompanying drawings, preferred embodiments ofthe present invention will be described below.

FIG. 1 shows a schematic structural drawing of a preferred embodiment ofthe present invention.

First, content of interest (101) rendered by media is input to both aninformation registering equipment (102) and an information viewingequipment (103). This content of interest is the same input for bothequipments.

This content of interest (101) corresponds to a keyword for aninformation registration and retrieval system. It may be anydistinguishable one for both equipments independently (that is, it isdistinguishable from another content rendered by media), such as a videoimage of a TV broadcast and an image from a Web site/page on theInternet.

If, for example, the content is rendered by TV broadcasting, asinformation to identify (105) the content of interest (101), thefollowing may be used: broadcast channel number, time when the contentwas broadcasted (year, month, day, hours, minutes, seconds, framenumber, etc.), receiving area (in the case of local TV broadcasting),etc. These specifics are obtained from the tuner of a TV receiver. Ifthe content is rendered from a Web site/page on the Internet, URL(Uniform Resource Locator) is used, instead.

On the assumption that content of interest (101) is rendered by TVbroadcasting, operation according to the preferred embodiment will beexplained below.

On the information registering equipment (102) shown in FIG. 1, thecontent of interest (101) is displayed and a video image reproduced fromthe content of interest (101) is linked to reference information (104)stored in a form such as a text document and HTML document on areference information server (100) that is provided internally in thesame registering equipment or as an external equipment.

During this operation, on the displayed image, target position/area aredefined with a coordinates pointing device (such as a mouse, tablet,pen, remote controller, etc.) included in the information registeringequipment (102).

As an example, defining target position/area by clicking on a skirt inthe displayed image is represented in FIG. 1.

The information registering equipment (102) transmits data consisting ofthe information to identify (105) the content of interest (101), thethus defined target position/area (106), and reference information (107)to which the content is desired to link across a computer network (108)to an information search equipment (109).

Hereon, the reference information (107) may be the same as the referenceinformation (104) stored in the reference information server (100). Ifthe content is to link to an open HTML document or the like on a Website/page accessible over the Internet, the reference information (107)may be a character string of URL (for example, http://www.???.co.jp/)that designates the address where the document exists. The characterstring of URL may be entered by the input means of the informationregistering equipment (102). If the reference information is a textdocument or the like, its designator may be entered by the input means,similarly.

On the other hand, on the information viewing equipment (103),similarly, the content of interest (101) is displayed. On the displayedimage, target position/area are defined with a coordinates pointingdevice included in the information viewing equipment (103). Searchingfor reference information (114) linked to the defined targetposition/area and outputting (presenting) the reference information aretried.

During this operation, as is the case for the information registeringequipment (102), the information viewing equipment (103) generates dataconsisting of the information to identify (112) the content of interest(101) and the thus defined target position/area (113) and transmits thedata across the computer network (108) to the information searchequipment (109), then waits for results of search to be returned fromthe information search equipment (109).

The information search equipment (109) shown in FIG. 1 is comprised ofan information matching apparatus (110) and a database (111). Theinformation search equipment (109) registers the data consisting of theinformation to identify (105) the content of interest, the definedtarget position/area (106), and reference information (107) to which thecontent is desired to link, received from the information registeringequipment (102), into the database (111) as a data record. At the sametime, the information matching apparatus (110) searches the database(111) for a data record that matches with the data consisting of theinformation to identify (112) the content of interest (101) and thedefined target position/area (113) received from the information viewingequipment (103). If there are matches between both the couple of theinformation to identify the content (105) (112) and between the coupleof the defined target position/area (106) (113), the registeredreference information (107) (if there are a plurality of items of thisinformation, a block of items) will be transmitted as the referenceinformation (114) across the computer network (108) to the informationviewing equipment (103).

Data record examples and operation of the information matching apparatus(110) will be explained later.

As the information matching apparatus (110) and the database (111), ageneral database server can be used, and therefore a detailed drawingthereof is not shown.

On the information viewing equipment (103), the reference information(114) received from the information search equipment (109) is presentedon its display, thereby linking of target content and relatedinformation is implemented. An example of the reference information(114) will be explained later.

FIG. 2 shows a detailed structural drawing of the informationregistering equipment (102) and the information viewing equipment (103).The information registering equipment (102) and the information viewingequipment (103) can be embodied in substantially the same structure.Based on the instructions of a software program comprising the stepswhich will be described later, stored in a program memory (211), acontroller (212) controls the overall operation of the equipment.

When the content of interest (101) rendered by media is supplied throughthe input (204), it is first encoded to digital data by an encoder (205)under the controller (212).

For this encoding, methods in compliance with the ISO/IEC standards,such as Moving Picture Experts Group (MPEG) and Joint PhotographicExperts Group (JPEG), and moreover, Pulse Coded Modulation (PCM) that isa simple binary coding method are applicable. Because any of thecommonly well-known methods can be used for encoding, a drawing thereofis not shown. During encoding, not only video signals, but also audiosignals may be encoded in the same way.

Encoded signals are decoded by a decoder (209) when information ispresented on the display (210). In addition to decoding the encodedvideo/audio signals, the decoder is capable of interpreting andvisualizing an HTML document or the like to display the referenceinformation (114) which will be explained later. The equipmentconfiguration may include a time-shifting apparatus (206) into which theoutput from the encoder (205) is stored once. After a certain timeinterval, the time-shifting apparatus outputs the encoded data to thedecoder (209).

The time-shifting apparatus (206) is comprised of a recording medium(208) and a drive (207) for recording/reproducing data by driving themedium.

As the recording medium (208), a disc-form medium as shown (for example,a compact disc (CD), digital versatile disc (DVD), magneto-optical (MO)disc, floppy disc (FD), hard disc (HD)) may be used. In addition, atape-form medium (such as videocassette tape) and a solid-state memory(such as a flash memory) may be used.

Because the time-shifting apparatus (206) can be embodied by the use ofa general recording apparatus as is, the drawing thereof is not shown.How to use the time-shifting apparatus (206) will be explained later.

As for the input (204) and display (210), the corresponding functions ofother devices can be used instead of them (that is, they can be providedas attachments) Therefore, they may be excluded from the configurationof the information registering equipment (102) and the informationviewing equipment (103).

Means for identifying content (203) shown in FIG. 2 obtains and retainsinformation for identifying the content of interest (101) (channel, timewhen content was broadcasted, receiving area, etc.), based on the dataentered through the input (204) which will be explained later.

If the time-shifting apparatus is used, when video/audio signals arerecorded, the information to identity the content can also be recorded.When reproduction is performed, the information is transferred from thetime-shifting apparatus to the means for identifying content (203).

A manipulator (201) shown in FIG. 2 allows the user to define the targetposition (horizontal and vertical positions in pixels) and the targetarea (within a radius from the target position) on the image presentedon the display (210), the image to which reference information isdesired to link or whose reference information is desired to beretrieved, based on the coordinates data furnished with the pointingdevice which will be explained later. In the case of the informationregistering equipment (102), the manipulator allows for specifying thereference information (104) (107) (input via the keyboard, dragging anddropping the label or the like of an HTML document, and so on). Then,means for obtaining target position/area (202) encodes the data on thetarget position/area set by the manipulator (201) and retains theencoded data. If the time-shifting apparatus (206) is used, defining thetarget position/area and specifying the reference information may becarried out when video/audio signals are reproduced.

The operation of a network interface (213) shown in FIG. 2 will bedescribed below.

The information registering equipment (102) transmits the information toidentify (105) the content of interest retained by the means foridentifying content (203), the target position/area (106) retained bythe means for obtaining target position/area (202), and the referenceinformation (104) (107) specified by the manipulator (201) over thecomputer network (108). On the other hand, the information viewingequipment (103) transmits the information to identify (112) the contentof interest retained by the means for identifying content (203) and thetarget position/area (113) retained by the means for obtaining targetposition/area (202) over the computer network (108) and waits forresults of search returned. When results of search are returned from theinformation search equipment over the computer network (108), theinformation viewing equipment (103) presents the results on the display(210). In either case, the network interface (213) only provides thefunctions of transmitting and receiving commands and data over thecomputer network (108) to/from the information search equipment. Becausethe network interface (213) can be embodied by the use of a networkinterface board or-the like for general PCs, a drawing thereof is notshown.

FIG. 3 shows a structural drawing in another form of the informationregistering equipment (102) and the information viewing equipment (103).

FIG. 3 illustrates an example of general PC-form embodiment of theconfiguration of the information registering equipment (102) and theinformation viewing equipment (103) shown in FIG. 2. The equipment iscomprised of a monitor (301), speakers (302-1) (302-2), a computer(303), mouse (304), drive (308), recording medium (310), keyboard (309),video capturing board (311), and network interface board (312).

By contrast with the configuration shown in FIG. 2, the equipment shownin FIG. 3 is configured with the mouse (304) and keyboard (309) in placeof the manipulator (201), the video capturing board (311) in place ofthe input (204), encoder (205), and decoder (209), the monitor (301) andspeakers (302-1) (302-2) in place of the display (210), the drive (308)in place of the drive (207), the recording medium (310) in place of therecording medium (208), and the network interface board (312) in placeof the network interface (213). Instead of the means for obtainingtarget position/area (202) and the means for identifying content (203),the main memory built in the computer (303) is used to retain theresults of operations executed by the controller (212). As the usermoves the mouse (304), the cursor shown on the monitor (301) moves to adesired place. By pressing, for example, the left button (305) of themouse, the target position is defined. Defining the target area can beperformed as follows. Move the wheel (307) of the mouse forward (up) toextend the area and move the wheel backward (down) to diminish the area.The right button (306) of the mouse may be used to display help forexplaining operation.

The video capturing board (311) is equipped with a TV tuner function sothat channel selection and tuning can be performed under the softwarecontrol from the computer (303). Thus, the information to identify (105)(112) the content of interest can be obtained easily. Other forms ofequipment configuration than the one shown in FIG. 3 include a TVreceiver or the like into which the functions shown in FIG. 2 areincorporated.

FIG. 4 illustrates a data record example used for information searchaccording to the present invention. This exemplary data record is storedinto the database 111 in the information search equipment (109) shown inFIG. 1.

The data record consists of fixed-length data which is, for example, atotal of 512 bytes. The record has the fields of descriptor (401),channel (402), time when content was broadcasted (403), receiving area(405), target position (406), target area (407), and referenceinformation (409). The reserves fields (404) (408) are provided forfuture extension of function. The data size of the fields and the totaldata size are illustrative for operation explanation that will be givenbelow and are not limited to the size given in FIG. 4 andvariable-length data fields rather than the fixed-length ones may beapplied.

In FIG. 4, the descriptor (401) is data indicating that the contents andformats of the fields that follow are those specified in FIG. 4; it isset at, for example, a binary value of “00000000” (fixed). If thecontents and formats of the fields that follow the descriptor (401)differ from those specified in FIG. 4, a different value of thedescriptor is set. In the fields of channel (402), time when content wasbroadcasted (403), receiving area (405), the contents of the informationto identify (105) the content of interest, which have been received fromthe information registering equipment (102), are stored as is. In thefields of target position (406) and target area (407), the targetposition/area (106), which have been received from the informationregistering equipment (102), are stored as is. In the field of referenceinformation (409), the reference information (107), which has beenreceived from the information registering equipment. (102), is stored asis. The record may include a field to contain the information registrarID code or the like, which is not shown in FIG. 4.

FIG. 5 illustrates an example of matching operation by the informationmatching apparatus (110) shown in FIG. 1

For the data records stored in the form shown in FIG. 4 in the database(11), the information matching apparatus (110) compares their contentswith the information to identify (112) the content of interest and thetarget position/area (113) received from the information viewingequipment (113) for matching. If a data record that matches with bothitems is found, the reference information (409) stored in the datarecord is transmitted to the information viewing equipment (103). Duringthis comparison, as for the information to identify the content ofinterest, if the data specifics received from the information viewingequipment are completely identical to those in the appropriate datafields of a data record, it is judged that there is a “match.”

For comparison in terms of the target position/area, on the other hand,whether there is a match is judged by, for example, a consistency checkmethod that is described below. First, on an assumptive display (501),the target position (406) stored in a data record, shown in FIG. 4, isassumed to be target position 1 (502) with its two-dimensionalcoordinates (x1, y1). The target area (407), stored in the data recordis assumed to be target area 1 (504) with its radius being radius 1, r1.Similarly, the target position/area (113) received from the informationviewing equipment (103) are assumed to be target position 2 (505) withits two-dimensional coordinates (x2, y2) and target area 2 (506) withits radius being r2.

At this time, if the assumptive circles of both areas are overlappedeven in part, it is judged that there is a “match”; that is, both areconsistent.

Specifically, see an example of consistency checking (508) in FIG. 5.Here, comparison is made between squared values that represent thedistance between target position 1 and target position 2 (left-handmember of inequalities (508)) and a squared value that represents thesum of radius 1 and radius 2 (right-hand member of inequalities (508).If the left-hand member equals to or smaller than the right-hand member,it is judged that both are consistent. If the left-hand member isgreater than the right-hand member, it is judged that both areinconsistent. This judgment can easily be implemented by simplearithmetic circuits and a comparator whose illustrative drawings are notshown.

There is a possibility that the information registering equipment (102)and information viewing equipment (103) have different display screendimensions in which the same content of interest (101) is presented.Accordingly, the above two-dimensional coordinates (x₁, y₁) and (x₂, y₂)and radii r₁ and r₂ may be normalized to the display screen size of eachequipment, that is, these values are divided by the vertical andhorizontal dimensions of the screen and the obtained values be used.

FIG. 6 illustrates an example of information linking operation accordingto the present invention.

FIG. 6 represents the actions of the information registering equipment(102), information search equipment (109), and information viewingequipment (103) shown in FIG. 1 as steps (600) that are carried out withtime lags. First, in steps (601) and (602), the same content of interest(101) rendered by media is input to the information registeringequipment (102) and information viewing equipment (103) at the sametime. This does not mean that the content of interest (101) is inputonly at this moment. In other words, during continuos rendering ofcontent by media, when the content of interest (101) may be captured andinput at any time, the image display content at this moment is input forthe process for registering and viewing of reference information. Atthis time, a finite processing time is required for registering andviewing the reference information. In particular, in cases whererecording/reproducing are performed with the above-mentionedtime-shifting apparatus (206), the time allowed for processing may bevery long; e.g., a few days or longer.

Thus, generally, there occurs some difference between delay time 1 (603)after the input of the content of interest (101) to the informationregistering equipment (102) in step (601) until transmitting theinformation to identify (105) the content of interest, targetposition/area (106), and reference information (107) in step (605) anddelay time 2 (604) after the input of the content of interest (101) tothe information viewing equipment (103) in step (602) until transmittingthe information to identify (112) the content of interest and targetposition/area (113) in step (606).

In FIG. 6, the actions proceed, assuming that delay time 1 (603) isshorter than delay time 2 (604), that is, the process of registeringinformation is completed before the process of viewing the referenceinformation. In this case, on the information search equipment (109),for all data records stored in its database (111), their data contentsare compared one by one with the data received in step (606) formatching, according to the above-described method of matching. If aregistered data record matches the data received, in step (607), theinformation search equipment (109) transmits the reference information(114) derived from the data record to the information viewing equipment(103). Then, the reference information (114) can be displayed in step(607).

FIG. 7 illustrates another example of information linking operationaccording to the present invention.

FIG. 7 represents the actions of the information registering equipment(102), information search equipment (109), and information viewingequipment (103) shown in FIG. 1 as steps (700) that are carried out withtime lags. Inversely to the case in FIG. 6, by comparison of delay time1 after the input of the content of interest (101) to the informationregistering equipment (102) in step (701) until transmitting theinformation to identify (105) the content of interest, targetposition/area (106), and reference information (107) in step (710) anddelay time 2 (704) after the input of the content of interest (101) tothe information viewing equipment (103) in step (702) until transmittingthe information to identify (112) the content of interest and targetposition/area (113) in step (705). the actions in FIG. 7 proceed,assuming that delay time 1 (703) is longer than delay time 2 (704), thatis, the process of registering information is not completed before theprocess of viewing the reference information.

In this case, on the information search equipment (109), for all datarecords stored in its database (111), even if their data contents arecompared with the data received in step (705) for matching, no datarecord that matches the received data is found. Thus, in step (706), theinformation search equipment (109) transmits notice of unregistered data(801), which will be explained later, to the information viewingequipment (103). At this time, the notice of unregistered data (801) maybe transmitted by e-mail or the like.

FIG. 8 illustrates an example of notice of unregistered data (801) thatis presented on the display of the information viewing equipment (103).

As shown in FIG. 8, the notice of unregistered data 801 consists of amessage (802) that data for which you are finding is not registered yetand notification conditions (803) in which the user is allowed tospecify how the user will be notified of the data that may be registeredhereafter.

The notification conditions (803) include options, for example, (a) nonotification is needed even if data is registered hereafter, (b) notifyme of the data as soon as it is registered, and (c) notify me of theresult of search for matching every XX days or YY hours (where XX and YYare numbers). The user can select the appropriate one out of theseoptions and enter it in the answer field (804).

The information viewing equipment (103) transmits the notificationconditions (803) with a value entered in the answer field (804) to theinformation search equipment (109) in step (707) in FIG. 7.

If the option “(b) notify me of the data as soon as it is registered”has been set as the user preference on the notification conditions(803), the above-described matching is performed each time newinformation is transmitted from the information registering equipment(102) and registered into the database of the information searchequipment (103). When data is transmitted in step (710) in FIG. 7 andregistered as a data record that meets the conditions of search, theinformation search equipment transmits the reference information (114)derived from the data record to the information viewing equipment (103).Then, the reference information (114) can be displayed in step (712).

When transmitting the reference information (114), the informationsearch equipment notifies the user by e-mail that the referenceinformation (114) has been registered. At the same time, the equipmentmay transmit the reference information (114) as an attachment file tothe e-mail. If the option “(c) notify me of the result of search formatching periodically” has been set as the user preference on thenotification conditions (803), and if the data records in the databaseare again compared with the target data for matching, but no data recordthat matches the target data is found, in step (708) in FIG. 7, theinformation search equipment re-transmits the notice of unregistereddata (801) to the information viewing equipment (103). The informationviewing equipment (103) displays the notice of unregistered data (801)and transmits the notification conditions (803) to the informationsearch equipment (109) in step (709).

These steps (708) and (709) are repeated by required times until theoption, (a) no notification is needed, has been set as the userpreference on the notification conditions (803) or the targetinformation has been registered in step (710). Once the targetinformation has been registered in step (710), the information searchequipment (109) can find a data record that meets the conditions ofsearch and thus transmit the reference information (114) derived fromthe data record to the information viewing equipment (103). Then, thereference information (114) can be displayed in step (712).

FIG. 9 illustrates an example of the reference information (114)displayed on the information viewing equipment (103) of the presentinvention.

As shown in FIG. 9, the information to identify (112) the content ofinterest, the content of interest (101) rendered by media, and thedefined target position/area (901) are displayed to tell the viewer whatcontent of interest for which its reference information has beensearched.

With this display, a time scrolling mechanism (902) may be employed sothat a scene (motion video sequence) in several seconds before and afterthe specified point (time) can be displayed, which enables the viewer torecall the scene of interest more definitely and recognize the search ofits reference information. The motion video sequence can be read fromthe above-mentioned time-shifting apparatus (206).

Reference information items (107-1) (107-2) (107-3) exemplified in FIG.9 are the reference information (107) that has been registered by theinformation registering equipment (102). Assuming that a plurality ofreference information items are presented on the same display at a timeas in this example, as the reference information (107), displayingtitles in several words with URLs where their HTML documents (referenceinformation (104) have been stored is preferable to directly displayingtext documents or the like, because more information can be displayed ata time.

In cases where URLs are specified as the reference information (107),select the appropriate URL (103), then the HTML document that the URLpoints to is displayed on the information viewing equipment. Forselecting an URL and displaying its HTML document, the same function isapplied as the browser software for general PCs and therefore thedrawing thereof is not shown. If the registered items of referenceinformation (107) are so many that they cannot be displayed on one page,a page forward mechanism (107) and a page turn-back mechanism (903) maybe employed so that the user can let the display page go forward andbackward.

FIG. 10 illustrates an example of post-processing of information linkingoperation according to the present invention.

By carrying out the information linking operation shown in FIG. 6 orFIG. 7 (step (600) or step (700)), after the reference information (114)exemplified in FIG. 9 is displayed in step (608) or step (712),information (1001) that what item of reference information (107) hasactually been selected by the viewer from among the referenceinformation items (107-1), (107-2), (107-3) and the viewer ID code(1002), which will be explained later, are transmitted to theinformation search equipment (109) in step (1003) in FIG. 10. Theinformation search equipment (109) transmits viewer information (1004)to the information registering equipment (102) that registered theselected item of reference information (107) in step (1005)

Hereon, the viewer information is data representing the identity andattributes of the viewer who selected the item of reference information(107), comprising the address, name, and age of the viewer, telephonenumber, e-mail address, yearly income, hobby, etc.

By feeding back of the attributes of the viewer to the informationregistrar, this post-processing can enhance the effectiveness ofadvertising for a commercial product and has an effect that theregistrar's will to register information becomes stronger. Thus,transmitting the viewer information (1004) in the step (1005) may beperformed as a service for which the information registrar is charged afee.

FIG. 11 shows a system structural diagram of illustrative embodiment ofimplementing the post-processing illustrated in FIG. 10.

In this embodiment, as the first step, when initializing the informationviewing equipment (103) at the first time use or the like, the viewerregisters viewer information (1004) beforehand into a database (1101)for viewer information provided on the information search equipment(109). At this time, the viewer ID code (1002) is assigned to theinformation viewing equipment (103) from the information searchequipment (109).

By carrying out the information linking operation shown in FIG. 6 orFIG. 7 (step (600) or step (700)), the reference information (114) asexemplified in FIG. 9 is transmitted from the information searchequipment (109) to the information viewing equipment (103). When one ofthe reference information items (for example, (107-1)) is selected,information (1001) that what item of reference information (107) hasbeen selected by the viewer from among the items on the referenceinformation (114) and the viewer ID code (1002) are transmitted to theinformation search equipment (109). By viewer management means (1102) onthe information search equipment, the viewer information is retrievedfrom the database (1101) for viewer information and transmitted to theinformation registering equipment (102). Thereby, transmitting theviewer information (1004) described in the above-mentioned step (1005)can be implemented.

The viewer management means (1102) and the database (1101) for viewerinformation can be embodied by using general database functions andtherefore the drawing thereof is not shown.

FIG. 12 illustrates another example of a data record used forinformation search according to the present invention. The data recordexample shown in FIG. 12 includes a start time (1203) field replacingthe field of time when content was broadcasted (403) shown in FIG. 4 andan end time (1204) field replacing the reserved field (404) shown inFIG. 4. This data record structure makes it possible to capture motionvideo frames with a finite time interval, specify the target position(406) and target area (407), and set up a link from the specifiedpoint/area to reference information (409).

With this modification, the descriptor (1201) changes from the binaryvalue of “00000000” to another, for example, a binary value of“00000001” that indicates alteration to the data record contents.

Hereon, both fields of start time (1203) and end time (1204) contain thespecifics of year, month, day, hours, minutes, seconds, frame number.The values in these fields respectively indicate the start time and theend time of broadcasting the content that is desired to link toreference information (409).

With the start time and the end time specified, the values of the targetposition (406) and target area (407) may be set to represent the fullextent of display. This setting makes it possible to set linking suchthat, when the viewer selects any of the video frames from the starttime to the end time of a program, the video frame is linked to itsreference information (409).

To represent the full extent of display, the target position (406) andtarget area (407) may be defined by assigning special values (forexample, all bits containing “1”) to them.

FIG. 13 illustrates yet another data record example used for informationsearch according to the present invention.

The data record example shown in FIG. 13 includes one couple or more oftarget position (1306) and target area (1307) fields replacing thereserved field (408) shown in FIG. 4. This data record structure makesit possible to set different target positions (1306-1) (1306-2) andtarget areas (1307-1) (1307-2) for a plurality of successive framesstarting from the specified time when content was broadcasted (403),such as first frame information (1301) and second frame information(1302) shown in FIG. 13. This enables that, even if the subject displayposition moves in motion video sequence, linking to its referenceinformation (409) will follow the change in position. The descriptor(1301) contains a flag indicating this data record structure (forexample, a binary value of “00000010” and the number of successiveframes. In FIG. 13, the number of successive frames is designated by a5-bit binary value, but not limited to this; more bits may be assigned.

If frame information (1301) (1302) greatly increases so that it cannotbe contained in one record set, it may be divided into a plurality ofrecord sets and stored into the database (111).

FIG. 14 illustrates a further data record example used for informationsearch according to the present invention.

The data record example shown in FIG. 14 includes a billing information(1408) field on the assumption that the information registrar is chargedfor registering reference information (409) and the viewer is chargedfor viewing the reference information (409). The descriptor (1401) isaltered to indicate this data record structure.

The billing information (1408) field comprises, for example, thefollowing sub-fields: account number (1410), registering fee (1411),viewing fee (1412), and term of validity (1413). The account number(1410) is the account of the information registrar who registered thisdata record to receive and pay charges: for example, the registrar'scredit card number or the like is encrypted and stored in the accountnumber sub-field. The registering fee (1411) is a charge (that is, aninformation registering fee) that is to be paid from the above accountfor registering this data record and this is used for calculating aninformation value (1601) which will be described later. The viewing fee(1412) is a charge (that is, an information viewing fee) that the vieweris to pay to the registrar through the above account for viewing thereference information (409) from this data record. The term of validityis a term during which this data record is valid. At its expiry, thisdata record is deleted from the database (111) (that is, it is excludedfrom the data for which search is performed). A new business using thepresent invention can be implemented in such a manner that all or partof charges that should actually be paid by the registrar and the vieweris paid to a service provider that operates the information searchequipment (109). The billing information (1408) field does notnecessarily include all the sub-fields (1410), (1411), (1412), and(1413). These sub-fields may be selectively provided for application orthe like. The data record structures shown in FIGS. 4, 12, 13, and 14are independent and a data record may be composed of fields from two ormore record structures as required, wherein the bits (flag) of itsdescriptor should be set to designate its structure.

FIG. 15 shows a structural diagram illustrating another illustrativeembodiment of the information search equipment (109) of the presentinvention.

In FIG. 15, the reference information output from information matchingapparatus (110) is displayed in the form shown in FIG. 9. To thisoutput, its display form is arranged by using reference informationordering means (1501) and reference information layout means (1502) togenerate new reference information items (1701) (1901). The referenceinformation items (1701) (1901) generated after the arrangement will beexplained later.

By the reference information ordering means (1501) shown in FIG. 15, thereference information items input from the information matchingapparatus (110), sequenced as read from the database (111), are sorted(rearranged in another order) and output. The sorting ways include (a)sort registered data in time sequence, (b) sort data by value ofinformation (1601), and (c) sort data by frequency of reference.

By applying these ways of sorting, an information item regarded as moresignificant for the viewer can be placed near the top among the items tobe displayed and presented as the higher-priority one on the display.

Hereon, possible information items that could be regarded as moresignificant for the viewer are early and long registered referenceinformation, newly registered reference information, referenceinformation that the registrar registered with a high fee payment forits registration, reference information viewed by many people, and thelike.

Sorting way (a) can be implemented by attaching data, date/time ofregistration specified in a reserved field (408) or the like shown inFIG. 4 to data records. Sorting way (b) will be explained later (seeFIG. 16 and the explanation thereof). Sorting way (c) can be implementedas follows. A counter is provided in a reserved field (408) or the likeshown in FIG. 4 to indicate the number of times the data has beenreferenced. The counter is incremented by one, each time the data isreferenced. Each reference information item with its current value ofthe counter is input from the information matching apparatus (110) tothe reference information ordering means (1501).

The reference information layout means (1502) shown in FIG. 15 adjuststhe character font (size, color, twinkling, etc.) for describinginformation or attaches an icon (a small pictorial symbol) forattracting the viewer's attention to information, based on the priorityobtained from the reference information ordering means (1501).

For example, the information item on the top line is written withcharacters of the greatest font (greatest point) among the itemsdisplayed, and the lower the line, the font size of the item becomessmaller. Besides, the top line information is made bright in twinklingred and an icon that indicates “recommendation” or the like is attachedto the beginning of the top line. In this way, an information item thatis more significant is displayed in appearance that attracts theviewer's attention. Accordingly, it can be expected that the viewer willselect that information item for reference more frequently.

Conversely, such result can be expected that the registrar would pay ahigher fee for registering information so that the registeredinformation will be given higher priority, expecting that the viewerwill select it more frequently.

The reference information ordering means (1501) and referenceinformation layout means (1502) shown in FIG. 15 may be incorporated inthe information viewing equipment (103) instead of the informationsearch equipment (109). In this configuration, the information searchequipment (109) sends the information viewing equipment (103) thereference information (114) and information necessary for ordering (timesequence of registered data, value of information, number of times thedata has been referenced, etc.). This configuration would obviously havethe same effect as described herein and the drawing thereof is notshown.

FIG. 16 illustrates an example of defining the value of information(1601) used for the billing method of the present invention.

As shown in FIG. 13, it is possible to capture and link the content ofinterest (101) as video images on a plurality of successive frames toits reference information (107). As shown in FIG. 16, when capturing thevideo images, the target area 1 (504) set by the information registeringequipment (102) generally increases or decreases in size, according tothe size of the subject shown on the frames (1602-1) (1602-2) (1602-3)(1602-4) (1602-5) of the video content of interest (101).

Hereon, by defining information value (1601) =registering fee (1411)/(atotal of target areas 1 (504) on the frames), the registering fee perunit area can be calculated. The higher the information value (1601),the higher will be the value of the reference information. Theinformation can be expected to be more significant for the viewer.

The area used in this calculation may be replaced by the number ofpixels or the like constituting the target area 1 (504).

FIG. 17 illustrates an example of reference information (1701) output tothe information viewing equipment (103) of the present invention. Thedisplay content shown in FIG. 17 is an example of the result ofrearranging the reference information (114) illustrated in FIG. 9,according to the ordering of reference information items illustrated inFIG. 15.

In this example, the reference information items (107-2) (107-1) (107-3)are presented in this order after sorting. The information item on thetop line is written with the greatest characters among the itemsdisplayed, and the lower the line, the character size of the itembecomes smaller. Besides, a specific icon (1072) is attached to thebeginning of the top line. Due to the ordering in this way, it can beexpected that the viewer will select that information item for referencemore frequently.

FIG. 18 shows a schematic structural drawing of another preferredembodiment of the present invention.

In FIG. 18, information registering equipment (1801) replaces theinformation registering equipment (102) in the structural drawing of thefirst preferred embodiment shown in FIG. 1.

If the start time (1203) and the end time (1204) of broadcasting aprogram are specified and the values of target position (406) and targetarea (407) are set to designate the full extent of display as part ofthe contents of a data record as exemplified in FIG. 12, the data to beset in all fields described in FIG. 12 can be prepared before thecontent of interest (101) is actually broadcasted, and thereforereference information (107) can be registered prior to the broadcastingof the content.

In this case, as shown in FIG. 18, the information to identify (105) thecontent of interest, target position/area (106), and referenceinformation (107) should, preferably, be provided beforehand in the formof the data record exemplified in FIG. 12 and directly registered on theinformation search equipment (109) by sending them over the computernetwork (108).

FIG. 19 illustrates an example of reference information (1901) output tothe information viewing equipment (103) of the present invention. Thedisplay content shown in FIG. 19 is an example of the result of sortinginformation items, according to the ordering of reference informationitems illustrated in FIG. 15, wherein the reference information (117)items are sorted under previous registered information (1902),registered information to charge for (1903), and registered informationfree of charge (1904) and presented on the display.

The previously registered information (1902) indicates that thereference information items (107-4) (107-3) have been registered priorto the time of actual broadcasting of the content of interest and thatthe information registrar probably has known the contents of the programbeforehand. The registered information to charge for (1903) indicatesthat the information registrar has registered a reference informationitem (107-1) and paid the fee for registering the information. Theregistered information free of charge (1904) indicates that theinformation registrar has registered a reference information item(107-5) without paying the fee for registering the information.Presenting the reference information items sorted in this way makes itpossible for the viewer to evaluate the usefulness of information tosome extent before selecting an item of reference information (107).

Such result can be expected that the registrar would pay a higher feefor registering information so that the registered information will begiven higher priority, expecting that the viewer will select it morefrequently.

The above-described embodiments discussed illustrative cases where thecontent of interest (101) is rendered by general TV broadcasting usingtransmission media such as terrestrial broadcasting, broadcastingsatellites, communications satellites, and cables. The present inventionis not limited to these embodiments. In this invention, information(data) that is rendered in various modes is applicable, including motionand still video contents which are distributed across computer networkssuch as the Internet, motion and still video data for which where thecontent of interest is stored is definite as the information to identitythe content, for example, the address of a general Web site/page on theInternet, and so on.

While the target area (407) was assumed as a circle and its sizerepresented in terms of the radius length of, the circle to simplify theexplanation in the above embodiments, other forms of the area are alsoconceivable. For example, complex forms of the target area would benumerically represented by AND operation and OR operation to calculatethe length of the edges of a rectangle and a plurality of figures incombination. The thus obtained value would be set in the target area(407) field in the data record.

By applying the present invention, the following is made possible. Whenwatching a TV program, only by selecting a part or all of an image shownon the TV receiver, even without entering some characters, other sourceinformation related to the image will be retrieved from the database andpresented to the viewer.

As additional effect of the present invention, such informationrendering service as a business would be feasible in such a manner as toprofit by charging a fee for registering information and a fee forinformation retrieval.

While the present invention has been described above in conjunction withthe preferred embodiments, one of ordinary skill in the art would beenabled by this disclosure to make various modifications to thisembodiment and still be within the scope and spirit of the invention asdefined in the appended claims.

1-4. (canceled).
 5. An information viewing equipment which links contentof interest rendered by media to its reference information, comprising:means for capturing said content of interest; means for obtainingcontent-identifying information to identify the content of interest fromthe captured content of interest; means for transmitting thecontent-identifying information over a computer network; means forreceiving the reference information over the computer network; and meansfor outputting the received reference information.
 6. An informationviewing equipment according to claim 5, wherein: said content ofinterest rendered by media is video image information distributed by TVbroadcasting and said content-identifying information are at least abroadcasting channel number over which and time when the content was orwill be broadcasted.
 7. An information viewing equipment according toclaim 6, further comprising: means for obtaining target-defininginformation that designates a part or all of a video image of interestdistributed by the TV broadcasting; and means for transmitting thetarget-defining information joined with the content-identifyinginformation over the computer network.
 8. An information viewingequipment according to claim 5, further comprising: a time-shiftingapparatus for recording and reproducing said content of interest.
 9. Aninformation registering equipment which links content of interestrendered by media to its reference information, comprising: means forcapturing said content of interest; means for obtainingcontent-identifying information to identify said content of interest;means for obtaining reference information to join with thecontent-identifying information; and means for transmitting a set of thecontent-identifying information and the reference information over acomputer network.
 10. An information registering equipment as recited inclaim 9, wherein: said content of interest rendered by media is videoimage information distributed by TV broadcasting and said means forobtaining content-identifying information are at least a broadcastingchannel number over which and time when the content was or will bebroadcasted.
 11. An information registering equipment as recited inclaim 9, further comprising: means for obtaining target-defininginformation that designates a part or all of a video image in thecontent of interest distributed by the TV broadcasting; and means fortransmitting the target-defining information joined with thecontent-identifying information and the reference information over thecomputer network.
 12. An information registering equipment according toclaim 9, further comprising: a time-shifting apparatus for recording andreproducing said content of interest. 13-16. (canceled)
 17. Aninformation viewing equipment according to claim 6, wherein saidcontent-identifying information includes a receiving area if the TVbroadcasting is local TV broadcasting
 18. An information registeringequipment according to claim 10, wherein said content-identifyinginformation includes a receiving area if the TV broadcasting is local TVbroadcasting (restricted area broadcasting).
 19. An information viewingequipment according to claim 6, further comprising: a time-shiftingapparatus for recording and reproducing said content of interest.
 20. Aninformation registering equipment according to claim 10, furthercomprising: means for obtaining target-defining information thatdesignates a part or all of a video image in the content of interestdistributed by the TV broadcasting; and means for transmitting thetarget-defining information joined with the content-identifyinginformation and the reference information over the computer network.